feat(client): add microphone passthrough button to controls toolbar#620
Conversation
Add mic toggle button to the bottom controls bar that enables users to share their local microphone with the remote neko session via WebRTC. The server already supports microphone capture (capture.microphone.enabled) but the legacy client had no UI to trigger getUserMedia and send an audio track to the peer connection. Changes: - base.ts: Add enableMicrophone/disableMicrophone methods that call getUserMedia and addTrack/removeTrack on the RTCPeerConnection. Mic is cleaned up automatically on disconnect. - controls.vue: Add mic button (fa-microphone/fa-microphone-slash) between play/pause and volume controls with tooltip and error handling. - en-us.ts: Add i18n strings for mic tooltips and error dialog.
|
Thanks for the implementation. Although no features are planned for the current legacy system and there is no new one, I am considering adding at least check if the user is allowed to share his media (microhpone) and if the feature is even allowed on the system level. Further more, it exposes the lack of control for the microphone. As one user sharing his audio can be easily overriden by another one doing that. And the GUI is not reflecting that. Since there is no currenst state for the user that would show who is currently "hosting the audio". Or it could be tied to the controls, so only person that is hosting can share their audio and when they stop hosting it's cleared. But it creates some complications for implicit hosting behaviour. |
|
Hey, thanks for maintaining neko, been using it for a remote Zoom setup through a Cloudflare Tunnel and it's been solid. Appreciate the feedback on the PR and for pushing that io.EOF fix. I went ahead and tied the mic to the active host, button only shows when you're controlling, auto-disables when you lose control, so no more conflicts with multiple users. That was a good call. For the system-level and per-user permission checks (canShareMedia, capture.microphone.enabled), I looked into it but those aren't exposed to the legacy client through the translation layer, so there's not much I can do on the client side without server changes. The server already rejects the track if mic is disabled which is a decent fallback. On a separate note, I've been poking around the v3 client branch, got it building and tested against the current server, seems to work well. Would love to help get it closer to production if you're open to that. What's still left to do there? |
Yeah, that should be enough without intrducing server changes to the legacy system.
Actually the v3 client is now only a library with some debug UI that is similar to what we have right now. But I was considering giving up on Vue and switching to React, because of the breaking change of Vue2->Vue3. I can't trust it anymore, that we migrate everything and there might be Vue4 compeltly depecating our whole codebase, again. |
2 participants
Summary
The neko v3 server supports microphone capture via WebRTC (
capture.microphone.enableddefaults totrue), but the legacy client bundled with the Docker images has no UI to enable it. This PR adds a microphone toggle button to the bottom controls toolbar.Changes
client/src/neko/base.ts: AddenableMicrophone()/disableMicrophone()methods toBaseClientthat:navigator.mediaDevices.getUserMedia({ audio: true })to capture the user's micRTCPeerConnectionviaaddTrack()client/src/components/controls.vue: Add mic toggle button between play/pause and volume slider with:fa-microphone/fa-microphone-slashiconsclient/src/locale/en-us.ts: Addmic_on,mic_off,mic_errori18n stringsHow it works
onnegotiationneeded(already handled inbase.ts)Use case
This enables voice calls (Zoom, Google Meet, etc.) through the remote browser session — the user's physical mic audio is relayed into the container's virtual audio input device.
Testing
npm run build)